Search for: All records

We propose a sampling method based on an ensemble approximation of second order Langevin dynamics. The log target density is appended with a quadratic term in an auxiliary momentum variable and damped-driven Hamiltonian dynamics introduced; the resulting stochastic differential equation is invariant to the Gibbs measure, with marginal on the position coordinates given by the target. A preconditioner based on covariance under the law of position coordinates under the dynamics does not change this invariance property, and is introduced to accelerate convergence to the Gibbs measure. The resulting mean-field dynamics may be approximated by an ensemble method; this results in a gradient-free and affine-invariant stochastic dynamical system with desirable provably uniform convergence properties across the class of all Gaussian targets. Numerical results demonstrate the potential of the method as the basis for a numerical sampler in Bayesian inverse problems, beyond the Gaussian setting. 

Kolmogorov-Arnold Networks (KAN) \cite{liu2024kan} were very recently proposed as a potential alternative to the prevalent architectural backbone of many deep learning models, the multi-layer perceptron (MLP). KANs have seen success in various tasks of AI for science, with their empirical efficiency and accuracy demonstrated in function regression, PDE solving, and many more scientific problems. In this article, we revisit the comparison of KANs and MLPs, with emphasis on a theoretical perspective. On the one hand, we compare the representation and approximation capabilities of KANs and MLPs. We establish that MLPs can be represented using KANs of a comparable size. This shows that the approximation and representation capabilities of KANs are at least as good as MLPs. Conversely, we show that KANs can be represented using MLPs, but that in this representation the number of parameters increases by a factor of the KAN grid size. This suggests that KANs with a large grid size may be more efficient than MLPs at approximating certain functions. On the other hand, from the perspective of learning and optimization, we study the spectral bias of KANs compared with MLPs. We demonstrate that KANs are less biased toward low frequencies than MLPs. We highlight that the multi-level learning feature specific to KANs, i.e. grid extension of splines, improves the learning process for high-frequency components. Detailed comparisons with different choices of depth, width, and grid sizes of KANs are made, shedding some light on how to choose the hyperparameters in practice. 

Abstract We study the singularity formation of a quasi-exact 1D model proposed by Hou and Li (2008Commun. Pure Appl. Math.61661–97). This model is based on an approximation of the axisymmetric Navier–Stokes equations in therdirection. The solution of the 1D model can be used to construct an exact solution of the original 3D Euler and Navier–Stokes equations if the initial angular velocity, angular vorticity, and angular stream function are linear inr. This model shares many intrinsic properties similar to those of the 3D Euler and Navier–Stokes equations. It captures the competition between advection and vortex stretching as in the 1D De Gregorio (De Gregorio 1990J. Stat. Phys.591251–63; De Gregorio 1996Math. Methods Appl. Sci.191233–55) model. We show that the inviscid model with weakened advection and smooth initial data or the original 1D model with Hölder continuous data develops a self-similar blowup. We also show that the viscous model with weakened advection and smooth initial data develops a finite time blowup. To obtain sharp estimates for the nonlocal terms, we perform an exact computation for the low-frequency Fourier modes and extract damping in leading order estimates for the high-frequency modes using singularly weighted norms in the energy estimates. The analysis for the viscous case is more subtle since the viscous terms produce some instability if we just use singular weights. We establish the blowup analysis for the viscous model by carefully designing an energy norm that combines a singularly weighted energy norm and a sum of high-order Sobolev norms. 

We present REGLO, a novel methodology for repairing pretrained neural networks to satisfy global robustness and individual fairness properties. A neural network is said to be globally robust with respect to a given input region if and only if all the input points in the region are locally robust. This notion of global robustness also captures the notion of individual fairness as a special case. We prove that any counterexample to a global robustness property must exhibit a corresponding large gradient. For ReLU networks, this result allows us to efficiently identify the linear regions that violate a given global robustness property. By formulating and solving a suitable robust convex optimization problem, REGLO then computes a minimal weight change that will provably repair these violating linear regions.